Log-Quad divergence for Non-negative Matrix Factorization in multi-model prediction

Ryszard Szupiluk; Paweł Rubach

doi:https://doi.org/10.59139/ws.2024.05.2

Ryszard Szupiluk Szkoła Główna Handlowa w Warszawie, Instytut Informatyki i Gospodarki Cyfrowej, Polska / Warsaw School of Economics, Institute of Information Systems and Digital Economy, Poland. ORCID: https://orcid.org/0000-0002-4416-7131 , Paweł Rubach Szkoła Główna Handlowa w Warszawie, Instytut Informatyki i Gospodarki Cyfrowej, Polska / Warsaw School of Economics, Institute of Information Systems and Digital Economy, Poland ORCID: https://orcid.org/0000-0001-5487-609X Wiadomości Statystyczne. The Polish Statistician, vol. 69, 2024, 5, s. 14-24 Opublikowano online: 31 maja 2024 DOI https://doi.org/10.59139/ws.2024.05.2 Sposób cytowania: Szupiluk, R., Rubach, P. (2024). Log-Quad divergence for Non-negative Matrix Factorization in multi-model prediction. Wiadomości Statystyczne. The Polish Statistician, 69(5), 14–24. https://doi.org/10.59139/ws.2024.05.2.

325 Wyświetlenia 40 Pobrania

ARTYKUŁ

(Angielski) PDF

STRESZCZENIE

The aim of this paper is to present a new Non-negative Matrix Factorization (NMF) algorithm based on Log-Quad divergence, and to demonstrate its application to the separation of latent destructive components contained in prediction results in a multi-model approach. We provide an example of its application to a real economic problem, i.e. forecasting electricity consumption on the basis of information about hourly use of electricity in Poland in the period of 1988–1997. We evaluated and compared this method with other blind signal (source) separation techniques, such as Independent Component Analysis (ICA) and Algorithm for Multiple Unknown Signals Extraction (AMUSE). The results show that the NMF algorithm based on Log-Quad divergence has an interesting ability to improve predictions for small volumes of data.

SŁOWA KLUCZOWE

Non-negative Matrix Factorization, NMF, latent components identification, blind source separation, blind signal separation, prediction, ICA, AMUSE

JEL

C02, C50

BIBLIOGRAFIA

Amari, S. (1985). Differential Geometry of Statistical Models. In S. Amari (Ed.), Differential-Geometrical Methods in Statistics (pp. 11–65). Springer-Verlag. https://doi.org/10.1007/978-1-4612-5056-2_2.

Berry, M. W., Browne, M., Langville, A. N., Pauca, V. P., & Plemmons, R. J. (2007). Algorithms and applications for approximate nonnegative matrix factorization. Computational Statistics & Data Analysis, 52(1), 155–173. https://doi.org/10.1016/j.csda.2006.11.006.

Bregman, L. M. (1967). The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR Computational Mathematics and Mathematical Physics, 7(3), 200–217. https://doi.org/10.1016/0041-5553(67)90040-7.

Cardoso, J.-F., & Souloumiac, A. (1996). Jacobi Angles for Simultaneous Diagonalization. SIAM Journal on Matrix Analysis and Applications, 17(1), 161–164. https://doi.org/10.1137/S0895479893259546.

Cichocki, A., & Amari, S. (2002). Adaptive Blind Signal and Image Processing: Learning Algorithms and Applications. John Wiley & Sons. https://doi.org/10.1002/0470845899.

Cichocki, A., Lee, H., Kim, Y.-D., & Choi, S. (2008). Non-negative matrix factorization with ?-divergence. Pattern Recognition Letters, 29(9), 1433–1440. https://doi.org/10.1016/j.patrec.2008.02.016.

Cichocki, A., Zdunek, R., Phan, A. H., & Amari, S. (2009). Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis and Blind Source Separation. John Wiley & Sons. https://doi.org/10.1002/9780470747278.

Comon, P., & Jutten, C. (Ed.). (2010). Handbook of Blind Source Separation: Independent Component analysis and Applications. Academic Press. https://doi.org/10.1016/C2009-0-19334-0.

Csiszár, I. (1978). Information measures: A critical survey. In J. Kožešnik (Ed.), Transactions of the Seventh Prague Conference on Information Theory, Statistical Decision Functions, Random Processes and of the 1974 European Meeting of Statisticians held in Prague, 18 to 23 August 1974 (pp. 73–86). Springer. https://www.fuw.edu.pl/~kostecki/scans/csiszar1978.pdf.

Dhillon, I. S., & Sra, S. (2005). Generalized Nonnegative Matrix Approximations with Bregman Divergences. In Y. Weiss, B. Schölkopf, & J. Platt (Eds.), Advances in Neural Information Processing Systems, 18 (pp. 283–290). https://proceedings.neurips.cc/paper/2005/hash/d58e2f077670f4de9cd7963c857f2534-Abstract.html.

Févotte, C., Bertin, N., & Durrieu, J.-L. (2009). Nonnegative Matrix Factorization with the Itakura-Saito Divergence: With Application to Music Analysis. Neural Computation, 21(3), 793–830. https://doi.org/10.1162/neco.2008.04-08-771.

Lee, D. D., & Seung, H. S. (1999). Learning the parts of objects by non-negative matrix factorization. Nature, 401(6755), 788–791. https://doi.org/10.1038/44565.

Rutledge, D. N., & Jouan-Rimbaud Bouveresse, D. (2013). Independent Components Analysis with the JADE algorithm. TrAC Trends in Analytical Chemistry, 50, 22–32. https://doi.org/10.1016/j.trac.2013.03.013.

Szupiluk, R., & Rubach, P. (2020). Identification of Delays in AMUSE Algorithm for Blind Signal Separation of Financial Data. In L. Rutkowski, R. Scherer, M. Korytkowski, W. Pedrycz, R. Tadeusiewicz, & J. M. Zurada (Eds.), Artificial Intelligence and Soft Computing. ICAISC 2020. Lecture Notes in Computer Science (pp. 253–261). Springer. https://doi.org/10.1007/978-3-030-61534-5_23.

Szupiluk, R., Wojewnik, P., & Ząbkowski, T. (2007). Smooth Component Analysis as Ensemble Method for Prediction Improvement. In M. E. Davies, C. J. James, S. A. Abdallah, & M. D. Plumbley (Eds.), Independent Component Analysis and Signal Separation. ICA 2007. Lecture Notes in Computer Science (pp. 277–284). Springer. https://doi.org/10.1007/978-3-540-74494-8_35.

Tong, L., Soon, V. C., Huang, Y. F., & Liu, R. (1990). AMUSE: A new blind identification algorithm. In IEEE International Symposium on Circuits and Systems (pp. 1784–1787). https://doi.org/10.1109/ISCAS.1990.111981.

Wróć do: